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ABSTRACT 

Bayesian classification methods have been extensively used in a variety of image processing applications, including 
medical image analysis. The basic procedure is to combine data-driven knowledge in the likelihood terms with 
clinical knowledge in the prior terms to classify an image into a pre-determined number of classes. In many 
applications, it is difficult to construct meaningful priors and, hence, homogeneous priors are assumed. In this 
paper, we show how expectation-maximization weights and neighboring posterior probabilities may be combined 
to make intuitive use of the Bayesian priors. Drawing upon insights from computer vision tracking algorithms, 
we cast the problem in a tissue tracking framework. We show results of our algorithm on the classification of 
gray and white matter along with surrounding cerebral spinal fluid in brain MRI scans. We show results of our 
algorithm on 20 brain MRI datasets along with validation against expert manual segmentations. 
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1. INTRODUCTION 

Bayesian classification methods have been extensively used in a variety of image processing applications, including 
medical image analysis. The basic procedure is to combine data-driven knowledge in the likelihood terms with 
clinical knowledge in the prior terms to classify an image into a pre-determined number of classes. There is 
an extensive body of work which examines classification in this context using brain atlases and other spatial 
and shape priors on the medical image data. 1 3 Frequently, however, algorithms are required to perform in the 
absence of sufficient prior clinical knowledge and researchers revert to maximum-likelihood estimation which 
assumes uniform priors on the data. 

In Figure 1, a categorization is portrayed which depicts a grouping of classification strategies which operate 
in the absence of clinical prior information. On the left, we show our algorithm which makes use of the prior 
terms by casting the classification problem in a slice-by-slice iterative framework. In the middle, we show our 
algorithm with uniform priors in a volumetric non-iterative framework. On the right, we group all other 3D 
classification algorithms which use uniform priors. 

In this paper, we explore the use of minimal prior information. Simply stated, we wish to make use of the 
prior terms by incorporating useful information which can be entirely derived from the image header and data 
itself. The prior information we propose to use includes a combination of expectation-maximization weights and 
neighboring posterior probabilities. This will be described in greater detail in Section 2.3. 

Before introducing our tissue tracking algorithm, we will discuss other Bayesian classification algorithms in 
Section 2.1 and 2.2. Second, in Section 2.3, we will outline our tissue tracking algorithm. Third, in Section 3, we 
will outline our experiments on 20 brain MRI data-sets. And, finally, in Section 4, we will present our results. 

2. BAYESIAN CLASSIFICATION ALGORITHMS 

In the first two subsections, we present previous work on Bayesian classification algorithms. First, we present work 
by Haker et al. A which outlines the general structure of Bayesian classification for tracking applications. Next, 
we present similar work by other groups which has been adapted for volumetric medical image segmentation. 
Then, in the last subsection, we present our work which casts the medical image segmentation into a tracking 
framework. 
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Figure 1 . Medical Image Classification Strategies 


2.1. 2D Tracking Algorithms 

Several groups have previously proposed Bayesian classification algorithms. Most relevant to our work is the 
work by Haker et al on the tracking of objects in 2D time-lapsed sequences of Synthetic Aperture Radar (SAR) 
data. 4 

At each time step in the tracking framework, the algorithm classifies the given 2D image slice into N classes. 
The segmentation proceeds as follows: 

First, the image data is used to generate likelihood probabilities, p(V\C), for a class C and an intensity value 
V. Typically, a normal distribution is assumed on the data and the parameters of the distribution are estimated 
from the 2D slice. 

Next, prior probabilities are constructed. In, 4 it is suggested that good results can be achieved by setting 
the priors of the current slice to the posteriors of the previous slice. This has a smoothing effect across time and 
tends to help push through noise which may occur on slices in time. 

In the last step of the algorithm, the likelihood and prior probabilities are used to compute the posterior 
probabilities, p(C\ U), via Bayes’ Rule. Following the application of Bayes’ Rule, each of the N components of 
the posteriors are independently spatially smoothed using the anisotropic smoother of Olver et al 5 Teo et al 
showed that such a smoothing of the posteriors results in a more effective noise removal than a smoothing of the 
original data. 6 

Finally, the segmentation is completed by assuming that each pixel belongs to the class of maximum prob¬ 
ability. It is well known that this “Maximum a posteriori test” minimizes the probability of segmentation 
error. * Using this decision rule, a final segmentation or label-map is constructed from the smoothed posterior 
probabilities. 

2.2. Volumetric Medical Image Classification 

Previously, Teo et al. 7 presented results of volumetric Bayesian segmentation on brain MRI scans. Their Bayesian 
segmentation algorithm was used to detect the white/gray matter boundary. Their algorithm is very similar to 
the algorithm stated above in Section 2.1. However, instead of iterating through the slices, they simply classified 
the entire volume in one shot. Therefore, they assumed homogeneous priors. Like the algorithm above, they also 
smoothed the posteriors and applied a MAP decision rule to achieve segmentation. 

Tt is possible for one to assign various subjective costs for different types of segmentation errors and to then generalize 
this MAP test to minimize the average Bayes risk; however, we do not wish to penalize any type of error in particular, so 
the MAP test seems appropriate. 
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Figure 2. Tissue Tracking General Structure 


After applying the Bayesian classification algorithm, Teo et al. perform a surface growing morphology 
operation to grow gray matter uniformly across the white matter surface to a predetermined thickness. They 
explain the due partial volume effects it is difficult to achieve accurate gray matter segmentations in MRI using 
this technique. 

In other related work, Yang et al. 8 developed a Bayesian segmentation algorithm for the segmentation of 
coronary arteries. In this case, homogeneous priors are also employed. Furthermore, this algorithm works in 
conjunction with an active contour model to achieve the final result. 

2.3. Tissue Tracking 

In this subsection, we present the details of our tissue tracking algorithm. Specifically, we show how expectation- 
maximization weights and posterior probabilities may be combined to make intuitive use of the Bayesian priors. 
Afterwards, we show results of our algorithm on 20 brain MRI data-sets along with validation against expert 
manual segmentations. 

We proceed in a similar fashion as previously introduced in Section 2.1. However, since the data in not 
time-lapsed, we view the scan axis as the axis of tracking. In Figure 2, we present the basic idea. This algorithm 
works by iteratively sweeping through the image volume and classifying the volume slice-by-slice. The resulting 
classification is 3D due to the fact that information is passed between slices through the iterative process. 

As in the other algorithms, this algorithm depends upon the user choosing the number, TV, of predetermined 
classes into which the imagery is to be classified. As before, at each step of the iteration through the data-set, 
our algorithm classifies the given 2D image slice into N classes. The segmentation proceeds as follows, refer to 
Figure 3 for an overview: 

First, the image data is used to generate likelihood probabilities, p(V\C), for a class C and an intensity value 
V. We assume that the intensities in a given image represent samples from a Gaussian mixture model comprised 
of N clusters. The parameters for each Gaussian component of the mixture model can be estimated via the 
expect at ion-maximization (EM) algorithm. We use the EM algorithm presented by Bilmes et al. 9 to estimate 
the means and variances along with the component weights which provide a measure of the prior probability for 
each given class. 

Next, we construct the prior probabilities, p(C). In many applications, it is difficult to construct meaningful 
priors by bringing non-data-driven information into the segmentation framework, and thus uniform priors are 
often employed. We suggest that there are meaningful data-driven priors one may use in these situations. For 
instance, in 4 the authors reported success using the previous posteriors as the current priors when classifying a 
time sequence of SAR images. Our method provides a convenient way for one to combine the posteriors from 
previous medical image slices, call these Ppost(C ), with the expectation-maximization weights computed from 
the data, denoted by Pem{C ), on the current slice to form the prior probabilities on the current slice. 

We begin with an intuitive explanation. For a very small slice spacing, we expect a high degree of correlation 
from one slice to the next and thus would likely benefit from using the posteriors of the previous slice to guide the 
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Figure 3. Tissue Tracking Algorithm 

choice of priors for the next slice. On the other hand, for a large slice spacing, it is undesirable to use information 
from previous slices to guide the segmentation of the next slice. In this case, we could set our priors according 
to the weights derived from the expectation-maximization algorithm. Hence, we use slice spacing, call this Z, to 
set the contributions of Ppost(C) and Pem(C ) as follows: 


P(C) = e z ^ x Ppost{C) + (1 - e z ^ x )pem(C) 


(1) 






where A is a free parameter which allows the user to tune the weights appropriately. 

As before, in the last step of the algorithm, the posterior probabilities are smoothed and the MAP decision 
rule is applied to achieve the final segmentation. 

3. EXPERIMENT 

We have applied this algorithm to 20 normal brain MRI data-sets. We used publicly available data-sets from 
the Internet Brain Segmentation Repository (IBSR) offered by the Massachusetts General Hospital, Center for 
Morphometric Analysis. 10, 11 The IBSR data-sets are Tl-weighted, 3D coronal brain scans after having been 
positionally normalized. Manual expert segmentations for these data-sets are publicly available and represent 
the ground truth used in this work. 

Also documented on the IBSR website are the details for a comparison of 6 separate classification algorithms 
on these 20 datasets. They use an overlap metric for validation known as the Tanimoto coefficient 12 which will 
be discussed further in Section 4. Using the Tanimoto coefficient, Rajapakse et al. compared the results of 6 
separate algorithms (each operating in the absence of clinical prior information) on the classification of these 20 
datasets into gray matter, white matter, and CSF. These 6 algorithms include the following: Adaptive MAP, 
Biased MAP, Fuzzy C-Means, MAP, Maximum-Likelihood, and Tree-Structure K-Means. 

These 20 coronal brain scans are convenient for comparison because they span the spectrum of quality of 
imagery. The worst cases have low contrast and relatively large intensity gradients. The best cases are more 
recently acquired and result in better overlap scores. 

Our code was written using the Insight Toolkit (ITK). 13 ITK is quickly becoming the imaging software of 
choice for researchers in the medical imaging community and all of the functionality needed for this algorithm 
exists in the latest release of ITK (version 3.0.0). In addition to ITK, we relied heavily upon various tools to 
perform our experiment, such as 3D Sheer 14 and FreeSurfer. 15 

The IBSR data is anisotropic (1mm x 1mm x 3mm), so we set the parameter Z = 3. We experimented with 
a variety of settings for the parameter lambda and found that the best results were achieved by setting A = 3. 
This corresponds to using 36.7% of the neighboring posterior and 63.3% of the EM weights for our prior. 

4. RESULTS 

Here we compare the results of our algorithm on the 20 IBSR datasets with the 6 algorithms discussed above in 
Section 3. We validate the results of our algorithm against the expert manual segmentations using the Tanimoto 
coefficient. The Tanimoto coefficient is defined by 1 — {n\ -\-ri 2 — 27112 )/{pi +^2 — ^ 12 ) in which n\ is the number of 
voxels of a particular class (e.g. the number of voxel of gray matter) in manual result and n 2 is number of voxels 
of that same class for the algorithm result. 77,12 represents the number of voxels contained in the intersection of 
the particular class from manual result and the algorithm result. The metric ranges from 0 to 1 while 1 signifies 
identical results and 0 indicates poor results. While other popular validation methods exist, such as the DICE 
measure, 16 the Tanimoto metric represents a good measure of overlap and is convenient for comparing against 
the IBSR results. 

The results of our methods as compared to the 6 other algorithms is shown in Figure 4 for gray matter voxels 
and in Figure 5 for white matter voxels. As can be seen, our algorithm performs better than the competing 
algorithms, especially for low quality imagery (note that the cases are ordered from poor quality on the left to 
high quality on the right). 

On average, the tissue tracking algorithm achieves a Tanimoto coefficient of 0.6236 for gray matter compared 
to 0.5609 for the 3D volumetric version and to 0.5254 for the other methods shown in the plot. Likewise, the 
tissue tracking algorithm achieves a Tanimoto coefficient of 0.6578 for white matter compared to 0.6039 for the 
3D volumetric version and to 0.5580 for the other methods shown in the plot. 

In Figures 6, 7, and 8, we show snapshots of our results on various data-sets and slices. In the left column, 
we show the original grayscale slice. In the middle column, we show the results of our classification algorithm. 
In the right column, we show the expert manual segmentation results. Note that cases 5-8 and 4-8 were poor 
quality cases while case 11-3 was a high quality case. 
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Figure 4. Overlap of Gray Voxels for Each Brain Scan 


5. CONCLUSION 

We have presented an algorithm for the Bayesian segmentation of imagery by casting the problem in a tracking 
framework and using priors derived from the data. Using slice thickness, we have proposed a combination 
of expectation-maximization weighting and previous posterior updating to construct meaningful priors. The 
application of this algorithm to brain MRI scans has shown positive results. 
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Figure 6. Case 5-8, Slice 36 
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Figure 7. Case 4-8, Slice 12 
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